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FOREWORD 

This  report  documents  research  performed  by  the  author  while  In  a 
Long-Term  Full-Tlirfe  Training  agreement  with  the  Air  Force  Systems  Command. 
The  objective  of  this  effort  was  to  evaluate  alternative  keying  logics 
for  the  entry  of  alphanumeric  data  Into  aircraft  subsystems. 

The  work  was  performed  In  support  of  Project  2403  "Flight  Control 
Technology",  Task  04,  "Control /Display  for  Air  Force  Aircraft  and  Aerospace 
Vehicles",  under  Work  Unit  24030411  "Workload  Problem  Assessment"  of  the 
USAF  Wright  Aeronautical  Laboratories,  Flight  Dynamics  Laboratory.  Mr. 
Larry  Butterbaugh  of  the  Crew  Systems  Development  Branch,  Flight  Control 
Division,  was  the  principal  Investigator  as  well  as  author. 

This  report  documents  work  performed  during  the  period  from  April  1980 
to  April  1981 . 

Special  thanks  Is  extended  to  Mr.  John  Kozina  and  Mr.  David  Mott, 
both  employed  by  the  Bunker  Ramo  Corp.,  for  their  hard  work  In  fabricating 
the  keyboards  and  microcomputer,  as  well  as  developing  the  Logic  programs. 
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AACC 

ASTIME 

ATIME 

CLTIME 

LACC 

LTIME 

NACC 

NTIME 

TRANACC 


GLOSSARY 


Alphabetic  Accuracy  >  mean  keying  accuracy  for  alphabetic 
characters. 

Alphabetic  Stroke  Time  -  mean  stroke  time  for  correctly 
keyed  alphabetic  characters. 

Alphabetic  Time  -  mean  keying  time  for  correctly  keyed 
alphabetic  characters  (1,2,  or  3  strokes). 

Calculated  List  Time  -  mean  calculated  list  time  for 
correctly  keyed  alphabetic  and  numeric  characters. 

List  Accuracy  -  mean  list  keying  accuracy. 

List  Time  -  mean  list  keying  time. 

Numeric  Accuracy  -  mean  keying  accuracy  for  numeric 
characters. 

Numeric  Time  -  mean  keying  time  for  numeric  characters. 

Transition  Accuracy  -  mean  keying  accuracy  for  alphabetic 
to  numeric  transitions. 
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SECTION  I 
INTRODUCTION 


1 .  BACKGROUND 

Numeric  keyboards  have  been  utilized  for  many  years  in  some  aircraft, 
such  as  the  C-5A  and  F-1110;  however,  these  keyboards  are  typically  part 
of  a  specific  aircraft  subsystem  (e.g.,  navigation  system)  and  therefore 
limited.  In  usn,  to  that  subsystem.  The  current  technology  trend  of 
using  digital  electronics  In  the  design  of  airborne  avionics,  however, 
has  not  only  created  the  potential  for  an  entirely  new  look  In  aircraft 
crew  station  design  (Figure  1),  but  also  established  the  capability  for 
Integrated  control  of  multiple  subsystems  through  the  use  of  a  single 
Integrated  alphanumeric  keyboard.  Such  a  keyboard  can  have  both  advantages 
and  disadvantages.  A  single,  alphanumeric  keyboard  could  serve  the  purpose 
of  several  currently  used  control  heads,  and  thus  reduce  the  space 
requirements  needed  for  Installing  the  standard  complement  of  navigation 
and  communication  control  boxes,  for  Instance.  Also,  cost  savings  could 
be  realized  through  the  reduction  of  hardware  components  and  installation 
time  associated  with  dedicated  subsystem  controls.  On  the  other  hand, 
such  a  keyboard  will  significantly  impact  crew  procedures,  and  require 
utilization  by  the  flight  crew  throughout  the  flight.  Table  1  scopes 
potential  subsystems  that  could  be  Interfaced  with  an  alphanumeric 
keyboard,  and  the  flight  seyments  during  which  the  keyboard  could 
find  use. 


In  response  to  the  potential  advantages,  subsystems  are  appearing 
on  the  market  which  have  Incorporated  a  full  alphanumeric  keyboard 
(Figures  2,  3,  4).  Such  subsystems,  at  present,  are  primarily  integrated 
navigation  systems  which  require  the  input  of  the  alphabetic  navigation 
aid  identifiers  (i.e.,  air  route  Intersections)  In  addition  to  the 
numeric  Input  of  latitudes  and  longitudes. 
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An  alphanumeric  keyboard  for  these  systems  could  be  Implemented 
In  the  form  of  the  "QWERTY"  (typewriter)  design,  If  there  weren't  space 
and  operational  constraints  applicable  to  keyboard  Implementation  In 
aircraft. 


Space  limitations  In  aircraft  crew  stations  have  been  a  continual 
constraint  on  designers.  The  impact  of  this  constraint  on  keyboard 
design  Is  that  all  36  alphanumeric  characters  (26  alphabetic  +  10 
numeric)  usually  do  not  have  a  dedicated  key  for  their  Input.  As  a 
result,  multifunction  keys  are  being  utilized  in  keyboard  designs, 
producing  keyboards  with  12  keys  in  a  4  x  3  matrix  (Figures  5,  6). 

The  operational  constraint  applicable  to  keyboard  design  and 
Implementation  is  that  it  be  operable  with  one  hand,  while  wearing 
gloves,  and  based  on  the  desirability  to  locate  the  keyboard  between 
the  pilot  and  copilot.  It  must  be  operable  by  either  hand.  These  space 
and  operation  constraints  make  It  apparent  that  the  conventional 
alphanumeric  keyboard  (QWERTY)  is  not  feasible  and  further  research 
should  be  directed  toward  the  Implementation  of  another  type  of 
keyboard. 


Several  U.S.  Government  data  bases  were  searched  including  the 
Defense  Tecnnical  Information  Center  (formerly  the  Defense  Documentation 
Center)  and  the  Control /Display  Information  Center,  in  order  to  locate 
any  relevant  keyboard  research  funded  by  the  Department  of  Defense, 
the  Federal  Aviation  Administration,  the  National  Aeronautics  and  Space 
Administration,  or  other  U.S.  Government  agencies.  Also,  the  DIALOG 
data  base  was  searched  for  relevant  research  reported  in  technical 
journals  or  in  conference  proceedings. 


These  data 
such  systems  in 
to  the  entry  of 
1 )  and  Reising, 


searches  revealed  that  up  to  now,  the  evaluation  of 
the  context  of  subsystem  operation  has  been  restricted 
numeric  characters.  Bateman,  et.  al .  (1978)  (Reference 
et.  al .  (1977)  (Reference  2)  evaluated  a  system 


Figure  5.  4x3  Matrix  Keyboard;  Top-to-Bottom  I 

(Telephone)  Arrangement  ] 
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Incorporating  a  numeric  keyboard  (telephone  configuration)  but  did  not 
report  performance  related  to  the  keyboard.  Deinlnger  (1960)  (Reference 
3)  studied  16  numeric  key  arrangements  and  found  small  dlfferen  °s,  in 
keying  speed,  between  the  now  standard  telephone  arrangement  and  the 
calculator  arrangement  (.70  second/character  for  the  telephone  vs.  .73 
second/character  for  the  calculator).  This  same  study  failed  to  find 
a  difference  In  e-ror  rates,  also.  Lutz  and  Chapanls  (Reference  4) 
evaluated  the  telephone  arrangement  against  the  calculator  arrangement 
and  reported  that  telephone  arrangment  is  "expected"  on  a  numeric 
keyboard  with  a  frequency  of  5  to  1 .  Alden,  et.  al .  (1972)  (Reference 
5)  reviewed  the  design  Issues  surrounding  alphanumeric  keyboards 
(e.g.,  key  displacement,  force,  etc.)  as  well  as  the  results  of  various 
studies  Investigating  key  arrangements.  Of  these,  one  study  by  Paul, 
et.  al .  (1965)  (Reference  C)  evaluated  the  telephone  alphanumeric 
arrangement  against  the  calculator  arrangement  and  found  the  telephone 
arrangement  superior  both  for  alphanumerics  and  alphabetlcs  alone,  and 
no  difference  In  numeric  keying.  Conrad  (1967)  (Reference  7)  and  Conrad 
and  Hull  (1968)  (Reference  8)  evaluated  the  telephone  arrangement  against 
the  calculator  arrangement,  and  found  the  telephone  arrangement  to  have 
superior  speed  (.67  second/character  vs.  .73  second/character)  and 
accuracy  (.55%  errors  vs.  1.16%  errors).  Klemmer  and  Lockhead (1 962) 

(Reference  9)  studied  average  error  rates  for  keypunch  installations 
(calculator  arrangement)  and  found  values  ranging  between  .2%  and  .06%. 

In  related  research,  Devoe  (Reference  10)  (1967)  evaluated 
alphanumeric  keying  time  against  alternative  methods  of  data  entry  and 
found  the  keying  of  formatted  data  to  be  faster  than  all  but  printing. 
Also,  Neal  (1977)  (Reference  11)  studied  the  time  interval  between  key 
strokes,  which  was  found  to  be  on  the  order  of  .15  second.  Also  studied 
by  Neal  was  the  time  to  key  strings  of  characters  using  two-handed 
keyboards,  which  was  found  to  average  about  .25  second/character. 

Dean  (1969)  (Reference  12)  evaluated  the  effects  of  vibration  on  data 
entry  performance,  and  found  that  vibration  appeared  to  have  no  effect 
on  accuracy  but  did  increase  keying  time  by  as  much  as  .25  second/ 
character. 
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Otho  types  of  keying  have  also  been  studied.  Ratz  and  Richie 
(1961)  (Reference  13)  and  Selbel  (1962)  (Reference  14)  studied  one-handed 
choking  keyboards  but  reported  single-finger  key  presses  as  the  fastest. 

While  much  research  has  been  directed  toward  the  arrangement  o*. 
keyboards,  and  keying  versus  alternative  Inputting  methods,  research 
regarding  the  logic  associated  with  keyboards  is  lacking,  or  at  least 
not  reported.  This  is  very  disconcerting  In  light  of  the  marketed 
designs  and  their  obvious  differences  regarding  keying  logic. 

This  research,  therefore,  Is  an  Initial  Investigation  regarding 
the  various  logics  which  can  be  employed  to  key-in  alphanumeric 
characters.  Optimally,  all  associated  Issues  regarding  keying 
performance,  such  as  the  effect  of  flight  clothing  (l.e.,  gloves) 
and  the  effect  of  the  logics  on  a  simultaneous  tracking  task  will 
hwVe  to  be  addressed.  However,  this  research  focuses  on  the  study 
of  the  logics  themselves.  Therefore,  a  relatively  pure  experimental 
context  was  designed  In  order  to  study  the  performance  differences 
contributed  by  the  logics. 

2.  RESEARCH  QUESTION 

The  objective  of  this  research  was  to  evaluate  alternative  keying 
logics.  Specifically,  three  logics  compatible  with  the  4x3  matrix 
keyboard  and  a  logic  utilizing  an  8  x  6  matrix  of  single-function  keys 
were  examined  with  regard  to  their  effects  on  keying  performance. 

Aside  from  noting  any  performance  differences.  It  was  also  the 
intent  of  this  research  to  Isolate  why  there  are  differences  between 
logics.  This  was  accomplished  by  examining  the  results  for  any  relation 
to  logic  structure,  procedure,  etc. 
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3.  SIGNIFICANCE 

The  results  of  this  study  should  find  useful  application  In  the 
design  of  advanced  aircraft  subsystem  controls  which  very  likely  will 
utilize  alphanumeric  keyboards  for  data  entry  Into  navigation, 
communication,  and  other  subsystems.  While  Investigated  In  the 
context  of  a  navigation  data  entry  task  for  aircraft  applications, 
the  results  are  applicable  to  similar  tasks  In  other  contexts  requiring 
one-hand  alphanumeric  keying. 

4.  ORGANIZATION  OF  REPORT 

Having  established  a  void  between  human  performance  data  and 
keyboard  design  trends,  this  report  proceeds  with  a  description  of  the 
detail  of  the  experimental  procedure  (Section  II),  the  statistical 
results  associated  with  the  collected  data  (Section  III),  a  discussion 
of  the  factors  associated  with  these  results  (Section  IV),  and  concludes 
with  a  design  recommendation  and  further  research  recommendations 
(Section  V). 
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SECTION  Ii 

EXPERIMENTAL  PROCEDURE 


1 .  KEYBOARD  LAYOUTS 

Three  alphanumeric  keyboard  designs  were  used  to  evaluate  four 
logics  for  keying  alphabetic  and  numeric  characters.  Two  designs 
utilized  a  6-key  by  3-key  matrix  arrangement  similar  to  that  found 
on  push  button  telephones.  The  third  design  utilized  a  6-key  by  8-key 
matrix  arrangement.  Each  keyboard  provided  the  full  complement  of 
26  alphabetic  and  ten  numeric  characters,  as  well  as  six  special 
function  capabilities.  The  special  functions  Incorporated  into  these 
keyboards  were  similar  to  those  found  on  commercially  available 
navigation  management  systems,  and  Included  a  forward  space,  a  backward 
space,  a  clear  entry,  an  enter,  a  slash  (/),  and  an  alphabet  mode  key. 
The  three  keyboard  designs  are  shown  In  Figures  7,  8,  and  9.  For 
purposes  cf  standardizing  the  references  to  these  designs  throughout 
this  report,  these  designs  will  be  referred  to  as  Keyboard  A,  Keyboard 
B,  and  Keyboard  C,  respectively. 

Keyboard  A  (Figure  7)  has  the  standard  top-to-bottom  numeric 
arrany  .ment,  w-  ich  has  been  found  superior  in  previous  studies  Alden, 
(R:fer^nce  5),  and  an  alphabetic  arrangement  that  has  been  proposed 
within  the  United  States  Air  Force  Aeronautical  Systems  Division  for 
Incorporation  into  a  standardized  avionics  integratea  control  system 
(Reference  15). 

Keyboard  B  (Figure  8)  Incorporates  an  arrangement  which  was 
developed  cs  an  efficient  computer  Interface,  while  retaining  the 
original  function  of  the  push  button  telephone  (Reference  16).  This 
design  has  as  its  foundation  the  frequency  distribution  of  English 
letters  and  minimizes  the  occurrence  of  the  most  frequent  letters 
(i.e.,  e,  t,  a,  o,  1,  n,  s,  h,  r,  d,  ,  .  ,)  in  the  middle  position 
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on  a  key.  (The  concern  for  their  location  Is  based  on  the  need  of  an 
additional  key-press,  In  order  to  specify  a  letter  In  the  middle 
position,  ever  the  two  key-presses  necessary  to  specify  a  letter  In 
either  the  left  or  right  position.  Refer  to  the  discussion  of  Logic 
3  In  this  section  for  further  details  regarding  the  keying  logic 
associated  with  this  design). 

keyboard  C  (Figure  9)  Is  a  design  functionally  separating  the 
alphabetic  and  nurerlc  keys.  This  layout  Incorporates  Individual  keys 
for  each  character,  which  Is  representative  of  commercially  available 
designs,  and  Is  Intended  for  one-handed  operation. 

For  purposes  of  the  study  reported  herein,  the  special  functions 
provided  on  the  keyboards  were  standardized  with  regards  to  operation 
and  location. 

2.  KEYING  LOGICS 

Four  alphabetic  character  keying  logics  (two  associated  with 
Keyboard  A,  one  associated  with  Keyboard  B,  and  one  associated  with 
Keyboard  C)  were  Identified  and  evaluated.  The  logic  associated  with 
keying  a  numeric  character  was  the  same  for  all  keyboards  and  will, 
therefore,  be  described  first. 

a.  Numeric  Character  Keying 

Numeric  character  keying  was  accomplished  by  selecting  the 
appropriately  labeled  key  on  the  keyboard,  and  depressing.  Repeated 
keying  of  a  particular  number,  or  keying  of  a  sequence  of  numbers 
required  a  key  depression  for  each  character. 

b.  Alphabetic  Character  Keying 

Alphabetic  character  keying  was  accomplished  by  one  of  four  logics. 
Each  logic  was  specific  to  a  keyboard  design  for  the  purposes  of  this 
research.  Logic  1  and  Logic  2  were  studied  using  Keyboard  A.  Logic  3 
was  studied  using  Keyboard  B.  Logic  4  was  studied  using  Keyboard  C. 
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(1)  Logic  1 


Alphabetic  characters  were  obtained  using  this  logic  by 
depressing,  in  sequence,  three  appropriate  keys.  The  first  key  In  the 


sequence  is  always  the  'alpha'  key  (this  Is  the 


alpha! 


key  In  Figure 


7).  The  second  and  third  key  depressions  specify  the  desired  letter. 

The  second  key  depressed  Is  the  key  which  has  the  desired  letter  In  Its 
alphabetic  subset  (e.g.,  the  '1'  key  In  Figure  7  has  on  It  the 
alphabetic  subset  of  'A,  B,  and  C').  The  third  key  depressed,  and 
the  one  which  specifies  the  Individual  letter  desired,  Is  the  key 
within  the  same  row  of  the  6-key  by  3-key  matrix  as  the  second  key, 
which  corresponds  to  the  left,  middle,  or  right  position  of  the 
desired  letter  within  Its  alphabetic  subset.  Illustrating  Logic  1 
(Figure  7),  creating  the  alphanumeric  string  ' 123BRAV0456'  would  consist 
of  the  following  sequence  of  key  depressions: 


ABC 

1 


1\ 


GHI 

3 


urn 


ABC 

1 


generates  the  number  ' 1 ' 

generates  the  number  ’2’ 

generates  the  number  '3' 

initiates  alphabetic  mode 

identifies  alphabetic  subset  (A,  B,  c) 

specifies  'B* 

initiates  alphabetic  mode 

identifies  alphabetic  subset  (P,  Q,  R) 

specifies  'R' 
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Identifies  alphabetic  subset  (A,  B,  C) 


specifies  'A' 


initiates  alphabetic  mode 

Identifies  alphabetic  subset  (V,  W,  X) 

specifies  'V' 

initiates  alphabetic  mode 

identifies  alphabetic  subset  (M,  N,  0) 

specifies  'O' 

generates  the  number  '4' 

generates  the  number  '5' 

generates  the  number  '6' 


(2)  Logic  2 

As  in  Logic  1,  alphabetic  characters  are  obtained  by 
depressing  in  sequence,  three  appropriate  keys.  The  first  key  in  the 
sequence,  as  It  Is  in  Logic  1,  is  always  the  'alpha'  key.  The 
second  key  in  the  sequence,  Is,  again,  the  key  which  has  the  desired 
letter  in  its  alphabetic  subset.  The  third  key  depressed  for  Logic 
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2,  however,  is  always  the  'V,  '2',  or  '3'  key,  depending  on  whether 
the  desired  letter  is  in  the  left,  middle,  or  right  position  of  the 
alphabetic  subset  identified  by  the  second  key  depression. 

Illustrating  Logic  2  (refer  to  Figure  7),  the  alphanumeric 
string  ' 1 23BRAV0456 1  is  created  by  the  following  sequence  of  key 
depressions : 


ABC 

I 


DEF 

2 


ALPHA 


ABC 

I 


DEF 

2 


(ALPHA  I 


jPQR 

6 


GHI 

3 


(ALPHAl 


ABC 

I 


ABC 

I 


lALPHAj 


generates  the  number  'I' 

generates  the  number  '2' 

generates  the  number  '3' 

initiates  the  alphabetic  mode 

identifies  alphabetic  subset  (A,  B,  C) 

specifies  'B' 

iniitates  alphabetic  mode 

identifies  alphabetic  subset  (P,  Q,  R) 

specifies  ' R 1 

initiates  alphabetic  mode 

Identifies  alphabetic  subset  (A,  B,  C) 

specifies  'A' 

initiates  alphabetic  mode 


•  / 
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Identifies  alphabetic  subset  (V,  W,  X) 
specifies  'V' 


initiates  alphabetic  mode 


identifies  alphabetic  subset  (u,  N,  0) 
specifies  'O' 


generates  the  number  '4' 


generates  the  number  '5' 


generates  the  number  '6' 


(3)  Logic  3 

Alphabetic  characters  are  obtained  using  this  logic  by 
depressing,  in  sequence,  either  two  or  three  keys,  depending  on  the 
position  of  the  desired  letter  within  an  alphabetic  subset.  This 
logic  has  two  'alpha'  keys  as  shown  in  Figure  8.  Letters  in  the 
left  or  right  position  within  an  alphabetic  subset  require  two 
sequential  key  depressions.  The  first  key  depression  is  either  the 
'ALPHA  L'  or  'ALPHA  R',  depending  on  whether  the  desired  letter  is  in 
the  left  or  right  position  within  the  alphabetic  subset,  respectively. 
The  second  key  depressed  in  the  sequence  of  two  is  the  key  that  has 
the  desired  letter.  The  middle  letter  in  an  alphabetic  subset 
requires  three  key  depressions,  in  sequence,  in  order  to  be  selected. 
The  first  and  second  key  depressed  is  always  the  'ALPHA  L'  and  'ALPHA 
R',  respectively.  The  third  key  depressed  in  this  sequence  is  the  key 
that  has  the  desired  letter  in  its  alphabetic  subset. 
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Illustrating  Logic  3  (refer  to  Figure  8),  the  alphanumeric 
string  ' 1 23BRAV0456 1  is  created  by  the  following  sequence  of  key 
depressions : 


generates  the  number  '1' 
generates  the  number  '2* 
generates  the  number  *3' 
Initiates  alphabetic  mode 


specifies  middle  letter  of  alphabetic  subset 


AB_  Identifies  alphabetic  subset  (A,  B,  C), 
4  generates  ' B ' 


initiates  alphabetic  mode,  specifies  left  letter 
of  alphabetic  subset 


identifies  alphabetic  subset  (R,  S,  T), 
generates  ' R ' 


■Initiates  alphabetic  mode,  specifies  left  letter 
of  alphabetic  subset 


Identifies  alphabetic  subset  (A,  B, 
generates  'A' 


), 


Initiates  alphabetic  mode 


specifies  middle  letter  of  alphabetic  subset 


UVW  identifies  alphabetic  subset  (U,  V,  W), 
6  generates  'V' 


U.PHA  initiates  alphabetic  mode,  specifies  left  letter 
I  of  alphabetic  subset 


identifies  alphabetic  subset  (0,  P,  Q), 
generates  'O' 


20 


AFWAL-TR-81 -31 04 


AB_ 

4 


generates  the  number  *4' 


generates  the  number  *5' 


generates  the  number  '6' 


(4)  Logic  4 


Alphabetic  characters  are  created  with  this  logic  by 
depressing  the  key  corresponding  to  the  desired  letter.  Illustrating 
this  logic  (Figure  9),  the  alphanumeric  string  '123BRAV0456'  1- 
created  by  depressing,  In  sequence,  the  following  keys: 


generates  the  number  ' 1  * 
generates  the  number  '2' 
generates  the  number  '3' 
generates  the  letter  ' B * 
generates  the  letter  * R ' 
generates  the  letter  'A' 
generates  the  letter  'V' 


generates  the  letter  ’O' 
generates  the  number  '4' 
generates  the  number  '5' 
generates  the  number  '6' 
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3.  TEST  HARDWARE 

a.  Keyboards 

The  three  keyboards  used  In  this  research  all  Incorporated  GRAYHILL 
Series  82  Single-Pole,  Push  Button  Switch  Modules.  These  switches  have 
a  total  travel  of  .130  Inch  (3.3mm),  a  travel  to  contact  of  .050  Inch 
(1.3mm),  and  an  operatlnq  force  of  4.0  ounces  (114. 3gm ) .  The  switches 
were  pre-assembled  by  GRAYHILL,  Inc.  into  6-key  modules  (2-key  by  3-key) 
with  a  center-to-center  distance  of  .687  inch  (17.4mm)  and  a  key 
separation  distance  of  .279  Inch  (7.1mm)  (Reference  17).  "The  modules 
were  mounted  In  a  sheet-aluminum  case  which  had  a  15-degree  upward 
slope,  front  to  back.  The  case  was  painted  black  using  Federal  Standard 
Number  595A  (Reference  18)  color  number  17038.  The  keys  all  had  color- 
coded  backgrounds  with  black  characters  overlayed.  The  'special' 
function  keys  on  all  keyboards  were  black  letters  (No.  17038)  (Reference 
18)  on  a  white  (No.  37875)  (Reference  18)  background.  Alphabet  labels 
on  the  keyboards  had  a  gold  (Reference  18)  (No.  33481)  background,  while 
the  numbers  had  a  yellow  (Reference  18)  (No.  33695)  background. 


Keyboards  A  and  B  had  Franklin  Gothic  Extra  Condensed  style 
characters  on  the  alphanumeric  keys.  Ten-point  characters  were  used 
providing  a  character  height  of  .1  inch  (2.5mm),  a  stroke  width  to 
character  height  ratio  of  1/5,  and  a  character  width  to  character 
height  ratio  of  1/2. 


Keyboard  C  had  Helvetica  Medium  style  characters  on  the 
alphanumeric  keys.  Fourteen-point  characters  were  used  providing  a 
character  height  of  .125  inch  (3.17mm),  a  stroke  width  to  character 
height  ratio  of  1/5,  and  a  character  width  to  character  height 
ratio  of  1/1 .25. 


The  characters  on  the  'special'  function  keys  were 
all  keyboards.  They  were  8-point  Franklin  Gothic  Extra 
characters  with  a  height  of  .08  inch  (2.03mm),  a  stroke 
character  height  ratio  of  1/5,  and  a  character  width  to 
height  ratio  of  1/3. 


the  same  for 
Condensed 
width  to 
character 
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b.  Cathode  Ray  Tube  (CRT)  Display 

An  RCA,  model  TL  1209,  black  and  white  monitor  was  used  to  provide 
the  subjects  visual  feedback.  The  display  area  was  seven  Inches  wide 
by  five  Inches  high  (17.7cm  by  12.7cm).  The  characters  displayed  were 
.25  Inch  high  (6.35mm),  had  a  stroke  width  to  character  height  ratio 
of  1/8,  and  a  character  width  to  character  height  ratio  of  1/1.6. 

c.  Microcomputer 

The  micro  computer  (Figure  10)  used  In  this  research  was  fabricated 
In  the  Flight  Dynamics  Laboratory  by  personnel  of  the  Bunker  Ramo 
Corporation.  The  backbone  of  the  microcomputer  was  a  Motorola  6808 
Read  Only  Memory  used  for  the  video  display  generator  executive  program 
(formatted  in  Kansas  City  Standard)  and  the  clock,  a  Motorola  6847y 
video  display  generator  (Debug  System;  TV-Bug  Version  1.2  by  Motorola- 
Austln,  Texas),  4  K  of  Random  Access  Memory  (RAM)  used  for  the  keyboard 
logic  software  Input,  the  printer  output  buffer,  and  the  displayed 
output,  and  input/output  ports. 

The  system  Is  diagrammed  In  Figure  11.  The  keyboard  logic  was 
stored  on  audio  cassette  tape  and  loaded  Into  the  RAM  for  each  system 
operation.  Output  displayed  on  the  CRT  was  repeated  on  a  Teletype 
model  3320  5JC  printer.  In  addition,  clock  time  was  printed  out  on 
the  Teletype  printer  as  predetermined  events  involving  the  keyboard 
occurred. 

4.  EXPERIMENTAL  DESIGN 

a.  Research  Question 

The  research  question  being  Investigated  is  whether  there  is  any 
difference  among  the  four  previously  discussed  keyboard  logics, 
as  measured  by  subject  performance.  In  particular,  it  was  the  intent 
to  evaluate  these  keyboard  logics  in  such  a  manner  that  a  single  logic 
might  be  recommended  for  aircraft  applications. 
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In  order  to  test  for  the  aforementioned  differences,  a  split-plot 
factorial  experiment  was  designed.  In  order  to  apply  this  design, 
the  following  assumptions  are  required  (Reference  19): 

•  Observations  are  drawn  from  normally  distributed 
populations 

•  Observations  represent  random  samples  from  populations 

•  The  population  variances  are  equal 

•  The  unbiased  estimators  of  the  population  variance 
(numerator  and  denominator  of  F  ratio)  are  independent 

The  layout  of  the  design  with  four  levels  of  Treatment  A  (Logics  1, 

2,  3,  and  4),  three  levels  of  Treatment  B  (Replications  1,  2,  and  3), 
and  seven  subjects  per  level  of  Treatment  A  Is  shown  In  Figure  12. 

b.  Independent  and  Dependent  Variables 

The  Independent  variables  were  the  four  keyboard/logic  combinations 
and  the  three  replications  of  the  task. 

The  dependent  variables  consisted  of  keying  time  and  keying 
accuracy  measures. 

c.  Subjects 

The  experimental  plan  called  for  28  subjects.  Three  female  and 
25  male  scientists  and  engineers  form  the  Air  Force  Wright  Aeronautical 
Laboratories'  Flight  Dynamics  Laboratory  participated  on  a  voluntary  basis. 
Each  subject  was  randomly  assigned  to  one  of  the  four  levels  of  the 
keyboard/logic  treatment.  None  of  the  subjects  had  prior  experience 
with  one-handed  alphanumeric  keyboards  of  the  type  studied.  All 
subjects  were  between  the  ages  of  25  and  55,  and  had  20/20  vision 
(corrected  or  uncorrected). 
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Figure  12.  Experimental  Layout 
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d.  Analysis  Strategy 

The  data  analysis  followed  the  progression  of  Figure  13  and 
permitted  examination  of  the  data  consistent  with  the  following 
research  questions: 


•  Is  there  a  Logic  effect  on  keying  performance? 

■  Which  performance  measures  (keying  speed  and/or  keying 
accuracy)  are  sensitive  to  the  Logics? 

•  What  Is  the  source  of  any  Logic  effect?  (e.g.  number 
of  keystrokes,  number  of  errors,  etc.) 

As  an  initial  step,  keying  time  and  keying  accuracy  for  numeric 
clusters  (a  string  of  consecutive  numeric  characters  only)  were  ana¬ 
lyzed.  Mean  numeric  keying  time  (NTIME,  the  mean  time  to  key  a 
single  numeric  character)  and  mean  numeric  accuracy  (NACC,  the  mean 
proportion  of  correctly  keyed  numeric  characters  to  total  numeric 
characters)  were  obtained  for  each  replication  of  each  logic  and 
subsequently  evaluated  graphically  and  statistically.  The  numeric 
data  were  used  to  test  the  experimental  assumption  that  the  subjects 
used  in  the  research  were  from  a  homogeneous  (i.e.,  that  nu  significant 
difference,  either  practical  or  statistical,  existed  between  subject 
groups). 

Keying  performance  for  the  total  list  was  examined  to  obtain  an 
overall  perspective  of  any  Logic  or  Replication  (learning)  effect. 

Mean  keying  time  (LTIME,  mean  time  to  key  the  entire  alphanumeric  list 
in  Figure  15)  and  mean  keying  accuracy  (LACC,  mean  proportion  of 
correctly  keyed  characters  to  the  total  number  of  characters)  were 
plotted,  and  statistically  analyzed  for  both  Logic  and  Replication 
effects . 


In  order  to  locate  any  Logic  or  Replication  effect  associated 
with  keying  alphabetic  characters,  mean  alphabetic  keying  time  (ATIME, 
the  mean  time  to  key  a  single  alphabetic  character),  mean  alphabetic 
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Figure  13.  Analysis  Strategy 
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stroke  time  (ASTIME,  the  mean  time  per  stroke  for  keying  an  alphabetic 
character),  and  mean  alphabetic  keying  accuracy  (AACC,  the  mean  proportion 
of  correctly  keyed  alphabetic  characters  to  total  alphabetic  characters) 
were  obtained  from  alphabetic  clusters  (a  string  of  consecutive  alphabetic 
characters  only)  and  analyzed.  The  ASTIME  parameter  was  analyzed  In 
addition  to  ATIME,  because  the  different  logics  required  a  different 
number  of  key-presses,  or  strokes,  to  create  an  alphabetic  character 
(refer  to  the  Logic  descriptions  In  Section  II).  Also,  ASTIME  Is  the 
alphabetic  parameter  equivalent  to  NTIME  for  numerics,  since  numeric 
characters  required  only  one  stroke  In  all  cases  (NTIME  is  the  mean 
stroke  time  per  numeric  character). 

In  addition  to  the  numeric  and  alphabetic  clusters,  there  were 
clusters  that  contained  an  alphabetic  character  followed  Immediately 
by  a  numeric  character  (e.g.,  8310  D67  or  FL200).  While  the  keying 
time  for  an  alphabetic/numeric  transition  was  imbedded  In  the  time  to 
key  in  the  entire  cluster  and  unextractable,  keying  accuracy  associated 
with  these  transitions  was  available.  Mean  transition  accuracy  (TRANACC, 
the  mean  proportion  of  correctly  keyed  characters  in  the  two-character 
transition  to  the  total  characters  In  all  transitions)  for  each  Logic 
and  Replication  was  obtained  and  statistically  analyzed  for  any 
resulting  effect. 

In  order  to  isolate  any  within-repl ication  learning  that  might 
be  occurring,  the  list  of  alphanumeric  clusters  used  for  this  research 
contained  repeats  of  selected  clusters  (e.g.,  ABC,  JUVTY,  495).  Mean 
values  of  AACC,  ASTIME,  NACC,  and  NTIME  across  subjects  were  obtained, 
for  both  the  initial  and  repeated  occurrence,  for  each  Logic  and 
Replication.  These  data  were  plotted  and  analyzed  for  any  significant 
learning  trend. 

In  addition,  other  analyses  of  the  data  were  performed  In 
isolated  cases  to  answer  specific  performance  related  questions.  These 
isolated  analyses,  as  well  as  the  other  analyses  mentioned  above  are 
discussed  in  more  detail  in  Section  III  and  Section  IV. 
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5.  PROTOCOL 

Each  subject  participated  In  one  session  which  consisted  of 
Instruction,  practice,  and  data  collection.  The  subjects  performed 
the  keyboard  task  seated  at  a  table.  The  CRT  was  directly  in  front 
of  the  subject  and  slightly  below  the  horizontal  line-of  sight.  The 
keyboard  was  placed  on  the  table  (recall  the  keyboard  had  a  built-in 
15°  angle  of  Inclination)  and  could  be  adjusted  fore  and  aft  to 
suit  the  subject's  comfort.  The  only  constraints  were  that  the 
keyboard  could  not  be  angled  to  the  side  and  that  it  had  to  stay 
in-line  with  the  subject's  right-arm  position,  paralleling  his 
centerline  to  the  CRT. 

The  experimental  task  was  to  key-in  a  list  of  alphanumeric  strings. 
Accuracy  was  stressed  over  speed;  thus,  the  subjects  were  permitted 
(and  instructed)  to  correct  errors.  Subjects  were  permitted  to  use 
only  one  hand  to  operate  the  keyboard,  their  right  one. 

a.  Instruction 

After  the  subject  was  seated,  he  was  first  asked  to  position 
himself  and  the  keyboard  as  described  above.  Instructions  were  then 
read  to  the  subject  (Appendix  A)  which  included  familiarization 
with  the  layout  of  the  keyboard  he  was  to  use,  and  instruction  as  to 
the  keying  logic  associated  with  the  keyboard.  The  instruction 
included  a  demonstration  by  the  experimenter  of  both  the  alphabetic  and 
numeric  Logics,  as  well  as  pre-training  practice  by  the  subject. 

Also  explained  at  this  time  was  the  format  of  the  typed  list  of 
alphanumeric  strings  the  subject  would  be  keying-in,  the  format  of  the 
CRT,  and  the  procedure/function  of  the  'special'  function  keys. 

Subjects  were  permitted  to  ask  questions  regarding  the  keyboard  Logic, 
display  format,  c r  task  procedures,  prior  to  the  formal  training  period. 

b.  Training 

After  a  short  break,  the  subject  was  presented  a  list  of  alphanumeric 
strings  for  training.  The  list  contained  all  alphabetic  and  numeric 
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characters  (36  total)  with  their  frequency  of  occurrence  as  near  to 
uniform  as  possible.  The  training  list  and  frequency  of  occurrence 
of  the  characters  Is  shown  in  Figure  14  and  Table  2,  respectively. 

Each  subject  trained  to  an  accuracy  criterion.  While  both  speed 
and  accuracy  are  relevant  criteria,  keying  accuracy  was  selected  because 
of  the  importance  of  accurate  entry  of  information  into  aircraft 
subsystems.  For  example,  an  error  in  entering  a  navigation  waypoint 
could  cause  the  aircraft  to  be  flown  off  the  desired  course,  with 
possibly  fatal  consequences.  When  the  subject  keyed  the  list  twice 
(in  succession),  correcting  any  errors,  he  was  assumed  to  have  learned 
the  logic  and  the  task  procedure. 

c.  Data  Collection 

The  subject's  experimental  session  concluded  with  the  collection 
of  test  data.  The  subject  was  reminded  that  accuracy  was  more  important 
than  speed,  once  again.  Subsequently,  the  subject  was  provided  a 
list  of  alphanumeric  strings  to  be  keyed  for  the  data  collection  task. 
The  list  was  formatted  identically  to  the  training  list,  but  was  longer. 
This  list  and  the  frequency  of  occurrence  of  the  alphabetic  and  numeric 
characters  is  shown  in  Figure  15  and  Table  3,  respectively. 

The  data  collection  was  completed  when  the  subject  had  completed 
his  three  replications.  The  time  between  replications  was  selected 
by  the  subject,  but  was  not  permitted  to  exceed  ten  minutes.  On  the 
average,  the  entire  session  with  a  subject  did  not  last  for  more 
than  1  hour  and  30  minutes. 

At  the  conclusion  of  the  experimental  session,  the  subject  was 
asked  to  complete  the  questionnaire  shown  in  Appendix  B. 
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Figure  14.  Training  List 
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COLUMN  A 
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TABLE  3 

FREQUENCY  DISTRIBUTION  OF  DATA  LIST  CHARACTERS 
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SECTION  III 
RESULTS 

Results  are  presented  for  keying  accuracy  measures  first, 
followed  by  keying  speed  measures.  Within  each  of  these  categories, 
the  results  are  presented  in  the  order  In  which  the  data  were  analyzed; 
that  is: 


•  Numeric  Performance 

•  Alphabetic  Performance 

•  Alphabetic/Numeric  Transition  Performance 
(accuracy,  only) 

•  Repeated  Cluster  Performance 

•  Estimated  List  Performance 

A  two-way  Analysis  of  Variance  (ANOVA)  was  used  as  the  test  for 
significant  differences  resulting  from  Logic  or  Replication  effects.  To 
use  this  ANOVA,  the  data  in  each  cell  of  Figure  12  were  reduced  to  a  mean 
value.  These  mean  values  were  then  used  to  conduct  the  ANOVA.  All 
ANOVAs  were  tested  at  the  a  =  .10  level. 

1 .  KEYING  ACCURACY 

The  results  reported  for  the  statistical  analysis  of  keying 
accuracy  were  obtained  using  an  arcsine  transformation  to  normalize  the 
collected  data. 

a.  Numerics 

The  mean  values  of  numeric  keying  accuracy  are  plotted  in 
Figure  16,  for  each  Logic  and  Replication.  Also,  1  sigma  ranges  are 
shown  for  each  mean.  A  two-sample  t-test  between  the  mean  numeric  keying 
accuracy  for  Logic  3  and  Logic  1  was  performed  in  order  to  affirm  the 
experimental  assumption  that  subject  groups  were  homogeneous.  The  data 
for  these  Logics  were  used  because  they  were  the  apparent  slowest  and 
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Figure  16.  Mean  Numeric  Keying  Accuracy  (Proportion 
Correct)  by  Logic  and  Replication 
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fastest,  and  no  statistical  difference  between  these  Logics  would  Imply 
a  lack  of  statistical  difference  between  any  of  the  Logics  (l.e.,  all 
numeric  performance  Is  equal).  The  results  of  the  t-test  procedure 
contradicted  this  assumption  by  finding  a  significant  difference  between 
Logics  1  and  3,  p  <  .008.  However,  although  Logic  1  appears  to  have 
consistently  poorer  performance  than  Logics  3  and  4,  and  poorer  perfor¬ 
mance  than  Logic  2  In  all  but  the  third  Replication,  from  a  practical 
viewpoint  the  Logic  had  no  effect  on  numeric  keying  accuracy.  This 
result  is  supported  by  the  difference  between  the  means  for  Logic  and  1 
and  the  other  Logics  which  Is  between  .001  and  .010,  for  a  difference 
amounting  to  between  1  error  in  1000  and  1  error  In  100.  Also,  the 
overlapping  distributions  provide  further  support  to  the  result  of  no 
practical  difference  for  mean  numeric  keying  accuracy.  A  two-way 
(Logic  x  Replication)  AN0VA  supports  this  interpretation  by  finding  no 
difference  between  the  Logics  with  regards  to  keying  accuracy. 

b.  Total  List 

The  total  list  mean  keying  accuracy,  with  1  sigma  range,  for  each 
Logic  and  Replication  Is  plotted  In  Figure  17.  Logic  4  showed 
consistently  the  most  superior  performance  and  Logic  1  showed  consistently 
orest  performance.  Total  list  keying  accuracy  for  Logics  2  and  3  are 
less  consistent.  A  two-way  (Logic  x  Replication)  AN0VA  showed  a 
significant  difference  between  Logic  means,  F[ 3,6)  ■  17.66,  p  <  .002, 

The  Duncan  Multiple  Range  Test  (a  =  .05,  df  =  6)  on  these  means 
revealed  Logic  1  was  significantly  different  from  Logics  2,  3,  and  4; 

Logic  4  was  significantly  different  from  Logics  1,  2,  and  3;  and  no 
difference  between  Logics  2  and  3. 

r  -,i..cs  of  a  two-way  (Replication  x  Subject)  AN0VA  testing  for 
a  Replication  effect  within  each  of  tne  Logics  revealed  no  signifi¬ 
cant  effect  for  any  of  the  Logics. 
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Figure  17,  Mean  Total  List  Keying  Accuracy  (Proportion 
Correct)  by  Logic  and  Replication 
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c.  Alphabetlcs 

Values  of  mean  alphabetic  keying  accuracy  for  each  Logic  and 
Replication  are  plotted  In  Figure  18,  along  with  the  1  sigma  range 
about  each  mean.  Consistently  superior  performance  Is  exhibited  by 
Logic  4,  with  the  poorest  performance  consistently  shown  by  Logic  1. 

The  data  for  Logics  2  and  3  are  not  as  consistent,  but  Logic  2  appears 
to  be  superior  to  Logic  3. 

A  two-way  (Logic  x  Replication)  ANOVA  confirmed  the  significant 
Logic  effect  F( 3,6)  *  20.46,  P  <  .001,  and  a  significant  Replication 
effect,  F (2,6)  -  4.10,  p  <  .075.  Duncan's  Multiple  Range  Test  (a  =  .05, 
df  =  6)  on  the  Logic  means  showed  Logic  1  significantly  different  from 
all  other  Logics,  Logic  4  significantly  different  from  all  other  Logics, 
and  no  difference  between  Logics  2  and  3. 

In  order  to  test  for  a  Replication  effect  within  each  Logic,  a 
two-way  (Replication  x  Subject)  ANOVA  was  performed  for  each  Logic. 

Lo  mcs  1,  3,  and  4  each  showed  no  significant  difference  across 
Replications.  Logic  2  had  a  significant  Replication  effect,  F{ 2,12)  = 
4.89,  p  <  .028.  A  Duncan's  Multiple  Range  Test  was  subsequently 
performed  on  the  Replication  means  for  Logic  2,  in  order  to  determine 
where  the  difference  occurred.  The  results  showed  a  significant 
difference  (a  ■  .05,  df  =  12)  between  Replication  1  and  Replications 
2  or  3.  Further,  the  results  showed  no  significant  difference  between 
Replications  2  and  3. 

d.  Alphabetic/Numeric  Transitions 

The  means  for  each  Logic  and  Replication  for  alphabetic/numeric 
transition  keying  accuracy  data,  along  with  the  1  sigma  range  about 
each  mean,  are  plotted  in  Figure  19.  A  two-way  (Logic  x  Replication) 
ANOVA  showed  no  difference  between  means  for  either  Logic  or 
Replication  effects. 
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e.  Repeated  Clusters  of  Characters 

With  regard  to  subject  performance  on  alphabetic  and/or  numeric 
clusters  that  were  repeated  in  the  data  collection  list,  mean 
numeric  keying  accuracy  and  mean  alphabetic  keying  accuracy  were 
extracted  for  the  initial  and  repeated  occurrences.  These  data  are 
plotted  for  each  Replication  by  Logic  in  Figures  20  and  21, 
respectively. 

Numeric  keying  accuracy,  while  exhibiting  different  trends 
across  Replications  for  each  of  the  four  Logics,  showed  no  practical 
significant  difference  within  the  Replication.  This  is  ascertained 
from  Figure  20,  where  the  data  points  correspond  to  either  zero 
errors  (proportion  correct  =  1.000)  or  one  error  (proportion  correct 
=  .952).  Another  result  apparent  in  Figure  20  is  the  consistency  cf 
numeric  keying  accuracy  for  Logics  3  and  4,  as  compared  with  either  Logic 

1  or  Logic  2. 

Alphabetic  keying  accuracy  for  repeated  clusters  also  exhibited 
various  trends  across  the  Replications,  Logic  1  showed  a  larger 
proportion  correct  on  the  initial  occurrence  of  the  repeated  cluster, 
for  each  of  the  three  Replications.  Also,  Logic  1  showed  an  improvement 
in  performance  across  Replications  for  the  repeated  clusters.  Logics 

2  and  3  showed  performance  that  tends  to  hover  between  a  proportion 
correct  of  .978  and  .989,  with  no  consistent  trend  across  Replications. 
Logic  4,  on  the  other  hand,  showed  consistent  performance  both  within 
and  across  Replications. 

f.  Summary 

Numeric  keying  accuracy  was  used  to  test  for  homogeneity  of  the 
subjects.  A  two-way  (Logic  x  Replication)  AN0VA  supports  the  homogeneous 
assumption,  as  does  visual  inspection  of  the  data.  Total  list  keying 
accuracy  shows  significant  differences  such  that  Logic  4  accuracy  >  Logic 
2  or  3  accuracy  >  Logic  1  accuracy.  Alphabetic  keying  accuracy  shows 
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Figure  21.  Mean  Alphabetic  Keying  Accuracy  (Proportion 
Correct)  on  Repeated  Clusters  for  all 
Replications  of  each  Logic 
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Identical  significant  differences.  Alphabetic/Numeric  Transition  keying 
accuracy  shows  no  significant  difference  among  Logics.  The  accuracy 
trends  of  clusters  of  alphanumeric  characters  repeated  within  a 
replication  are  inconclusive  regarding  within  replication  learning. 

2.  KEYING  SPEED 

The  measure  for  keying  speed  used  throughout  the  analysis  was  seconds, 
recorded  to  the  thousandth  of  a  second  (.001  second  or  1  millisecond). 

a.  Numerics 

Mean  numeric  keying  time,  along  with  the  one  sigma  range,  for  each 
Replication  of  each  Logic  is  plotted  in  Figure  22.  A  two-sample  t-test 
of  the  means  for  Logics  3  and  4  was  performed  in  order  to,  again, 
confirm  the  homogeneity  of  the  subjects.  The  result  showed  no  signifi¬ 
cant  difference  between  these  means,  which  can  be  interpreted  as  the  data 
for  four  Logics  are  from  the  same  population  (i.e.,  homogeneous  subjects). 

A  two-way  (Logic  x  Replication)  ANOVA  showed  a  significant  Replica¬ 
tion  effect,  F( 2,6)  =  14.12,  p  <  .005,  on  numeric  keying  time.  Each 
Logic  was  then  analyzed  by  a  two-way  (Replication  x  Subject)  ANOVA  in 
order  to  locate  which  Logics  had  Replication  effects.  The  results 
indicated  significant  effects  for  Logic  1 ,  F(2,12)  *  10.82,  p<  .002, 

Logic  2, F  (2,12)  =  14.62,  p  <  .006,  and  Logic  4,  F(2,13)  =  7.67,  p  <  .007. 
Duncan's  Multiple  Range  Test  was  performed  on  these  Logics  and  significant 
differences  (a  =  .05,  df  =  12)  in  performance  were  found  for  the  second 
versus  third  Replication  of  both  Logic  1  and  2.  Logic  4,  however,  showed 
no  difference  in  performance  for  the  second  versus  third  Replication. 

b.  Total  List 

Mean  keying  times  and  one  sigma  ranges  for  the  total  list  are 
plotted  in  Figure  23  by  Logic  and  Replication.  As  shown,  the  means 
for  Logics  1,  2,  and  3  are  almost  luentical  within  each  Replication. 

These  same  three  Logics  show  similar  improvement  across  the  three 
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Figure  22.  Mean  Numeric  Keying  Time  (Seconds)  by  Logic 
and  Replication 
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Replications.  Also,  easily  seen  is  the  consistent  superiority  of  Logic 
4,  which  shows  improvement  across  the  Replications  paralleling  that  of  the 
other  Logics.  A  two-way  (Logic  x  Replication)  ANOVA  confirmed  these 
results  as  significant  for  both  Logics,  F(3,6)  =  41.34,  p  <  .002,  and 
Replications,  F(2, 61  =  17.47,  p  <  .003.  Duncan's  Multiple  Range  Test 
on  Logic  means  produced  the  expected  results  of  no  difference  between 
Logics  1,  2,  and  3,  and  a  significant  difference  (a  =  .05,  df  «  6) 
between  these  three  Logics  and  Logic  4. 

In  order  to  further  analyze  the  Replication  effect,  two-way 
(Replication  x  Subject)  ANOVAs  were  performed  on  the  total  list 
keying  time  data  of  each  Logic.  All  four  Logics  exhibited  significant 
Replication  effects  with  F(2,12)  =  25.72  and  p  <  .001  for  Logic  1, 

F(2,12)  =  18.93  and  p  <  .002  for  Logic  2,  F(2,12)  =  10.48  and  p  <  .002 
for  Logic  3,  and  F(2,12)  =  14.29  and  p  <  .0007  for  Logic  4.  Duncan's 
Multiple  Range  Test  on  Logic  1  data  revealed  a  significant  difference 
between  each  Replication.  Duncan's  Test  on  Logic  2  produced  identical 
results,  that  is,  mean  total  list  keying  time  performance  on  each 
Replication  was  significantly  different  from  the  performance  on  the 
previous  Replication.  Duncan's  Test  on  Logic  3,  however,  revealed  a 
significant  difference  between  Replications  1  and  2,  and  no  difference 
in  the  performances  on  Replications  2  and  3.  Logic  4,  according  to 
Duncan's  Test,  showed  significant  differences  between  all  Replications 
for  mean  total  list  Keying  time. 

c.  Alphabet  ics 

Results  for  alphabetic  keying  time  are  presented  for  each  Logic, 
as  this  reflects  the  performance  differences  between  Logics  to  key- 
in  an  alphabetic  character.  This  parameter,  however,  is  biased  in 
favor  of  Logic  4  because  of  the  different  number  of  strokes  required 
to  key-in  an  alphabetic  character  (3  strokes  for  Logics  1  and  2, 

2  or  3  strokes  for  Logic  3,  and  1  stroke  for  Logic  4).  In  order 
to  remove  this  bias,  stroke  times  were  computed  for  each  Logic  and 
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subsequently  analyzed.  The  results  of  the  analysis  of  alphabetic 
stroke  time  are  also  presented. 

The  means  and  one  sigma  ranges  for  alphabetic  character  keying  time 
are  plotted  for  each  Logic  and  Replication  In  Figure  24.  As  anticipated, 
the  performance  of  Logic  4  Is  consistently  and  uniformly  superior  to  the 
other  Logics.  A  two-way  (Logic  x  Replication)  ANOVA  confirmed  the 
existence  of  a  significant  difference,  F( 3,6)  =  51.49,  p  <  .0001, 
resulting  from  a  Logic  effect.  Duncan's  Multiple  Range  Test  on  Logic 
means  showed  no  difference  between  Logics  1,  2,  and  3,  while  Logics  with 
regard  to  alphabetic  character  keying  time. 

The  previous  ANOVA  also  found  a  significant  Replication  effect, 

F{ 2,6)  =  11.46  and  p  <  .008,  Each  Logic  was  individually  analyzed  using 
a  two-way  (Replication  x  Subject)  ANOVA  ir,  order  to  determine  the 
presence  of  a  Replication  effect.  As  expected  from  Figure  24,  all 
Logics  had  significant  Replication  effects,  P(2,13)  =  23.24,  p  <  .0001 
for  Logic  1,  F(2,12)  =  20.00,  p<  .0002  for  Logic  2,  f( 2,12)  =  8.95, 
p  <  .004  for  Logic  3,  and  F ( 2 ,1 2 )  =•  16.40,  p  <  .004  for  Logic  4. 

Duncan's  Multiple  Range  Test  showed  significant  differences  (a  =  .05, 
df  =  12)  between  all  Replications  for  Logics  1,  2,  and  4.  Logic  3  showed 
a  significant  difference  (a  =  .05,  df  -  12)  between  Replications  1  and 
2  only,  however. 

The  means  for  alphabetic  stroke  time,  and  their  1  sigma  ranges, 
are  plotted  in  Figure  25  by  Logic  and  Replication.  As  the  Figure 
shows.  Logics  1  and  2  showed  the  smallest  stroke  times  across  all 
Replications,  with  Logic  4  consistently  the  slowest.  As  the  data 
suggest,  a  two-way  (Logic  x  Replication)  ANOVA  showed  both  a  Logic 
effect,  r(3,6)  =  23,00,  p  <  .001,  and  a  Replication  effect,  F ( 2 ,6)  = 
11.70,  p  <  .008.  Duncan's  Multiple  Range  Test  on  the  Logic  means 
produced  expected  results  of  no  significant  difference  between  Logics 
1  and  2  or  Logics  3  and  4.  A  significant  difference  (a  =  .05,  df  =  12) 
was  detected,  however,  between  these  pairs. 


50 


AFVJAL-TR-81  -3104 


A  la|k  1  Mull  ±1<riri  tllffl 

O  U|»  2 

x  Ifgic  3 

□  Lifie  4 

Figure  24.  Mean  Alphabetic  Character  Keying  Time 
(Seconds)  by  Logic  and  Replication 
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As  a  result  of  the  Replication  effect,  two-way  (Replication  x 
Subject)  ANOVAs  were  performed  on  the  data  fcr  each  Logic  in  order 
to  specify  this  effect.  From  this  analysis,  significant  Replication 
effects  were  found  for  Logic  1,  F( 2,12)  =  28,38,  p  <  .0001,  Logic 
3,  F(2,12)  =  14.48,  p<  .0006,  and  Logic  4,  F  (2,12)  =  16.40,  P<  .0004. 
Duncan's  Multiple  Range  Test  performed  on  these  Logics  revealed 
significant  differences  (a  =  .05,  df  =  12)  between  all  Replications 
for  both  Logics  1  and  4.  Logic  3  had  a  significant  difference  (a  = 

.05,  df  »  12)  between  Replications  one  and  two,  but  no  difference 
between  Replications  two  and  thre*. 

d.  Repeated  Clusters  of  Characters 

Mean  numeric  keying  time  and  alphabetic  stroke  time  for  both 
the  initial  and  repeated  occurrence  of  alphanumeric  clusters  are 
plotted  in  Figures  26  and  27,  respectively,  for  all  Logic  and 
Replications. 

The  data  for  numeric  keying  time  show  both  within-Repl ication 
and  across-Repl ication  improvement  for  Logics  2  and  3.  The  data  for 
Logics  1  and  4  show  across-Repl ication  improvement  for  numeric  keying 
time;  however,  within-Repl ication  performance  with  these  Logics  shows 
no  consistent  improvement.  Also  noticeable  is  the  magnitude  of  the 
across-Repl ication  improvement  for  Logics  2  and  3,  as  compared  to 
Logics  1  and  4.  These  differences  are  consistent  with  the  previously 
presented  data  or  mean  numeric  keying  time  for  all  numerics,  in  that 
Logics  1  and  4  had  shorter,  similar  keying  times  and  exhibited  a 
similar  across-Repl ication  Improvement.  The  previous  data  for  Logics 
2  and  3  are  also  consistent  with  these  across-Repl ication  results  in 
that  Logics  2  and  3  exhibited  similar  keying  times  which,  across- 
Repl  ications,  'pproached  but  did  not  achieve  the  keying  times  of 
Logics  1  and  4. 

The  data  for  alphabetic  stroke  time  associated  with  repeated 
clusters  also  exhibit  across-Repl ication  improvement  for  all  Logics. 
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Figure  2C.  Mean  Numeric  Keying  Time  (Seconds)  on 

Repeated  Clusters  for  all  Replications  of 
each  Logic 
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Figure  27.  Mean  Alphabetic  Stroke  Time  (Seconds)  on 
Repeated  Clusters  for  all  Replicetions  of 
each  Logic 
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Withln-Replicatlon  Improvement  of  alphabetic  stroke  time  Is  consistently 
present  In  Logics  2  and  4,  and  generally  present  In  Logics  1  and  3. 

These  data  also  are  consistent  with  the  previous  data  presented  for 
alphabetic  stroke  time  associated  with  all  alphabetic  clusters.  As 
before.  Logic  2  initially  had  the(  shortest  stroke  time,  but  was  out¬ 
performed  by  Logic  1  at  the  completion  of  the  session.  Also,  as  before. 
Logic  4  exhibited  the  slowest  stroke  time  for  each  Replication. 

e.  Summary 

Numeric  keying  time,  like  numeric  keying  accuracy,  was  used  to  test 
for  subject  homogeneity.  Statistical  analysis  of  the  data  confirms  this 
assumption,  as  does  visual  inspection.  Total  list  keying  time  data  show 
no  significant  difference  among  Logics  1,  2,  and  3,  but  the  data  do  show 
that  Logic  4  is  significantly  different  from  the  other  three.  Alphabetic 
character  keying  time  shows  identical  results.  Alphabetic  stroke  time 
data  shows  no^ difference  between  Logics  1  and  2  or  Logics  3  and  4.  A 
significant  difference  was  found  between  these  pairs,  (e.g..  Logic  2 
significantly  different  from  Logic  3),  however.  The  trends  in  keying 
time  for  alphanumeric  clusters  repeated  within  a  replication  generally 
show  some  improvement  of  keying  time  both  within  and  across  replications 
of  a  Logic. 
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SECTION  IV 

DISCUSSION  OF  RESULTS 

The  results  presented  previously  Indicate  both  keying  accuracy  and 
keying  speed  are  affected  by  the  Logic  condition.  Keying  speed 
appears  most  variable,  however,  which  was  expected  due  to  the 
instructions  given  the  subjects  to  key-in  the  list  accurately.  Also, 
the  results  show  differences  in  the  pattern  of  performance  improvement 
across  Replications,  which  varied  for  both  the  Logic  condition  and 
performance  measure  considered.  These  and  other  results  are  discussed 
in  this  section. 

Each  Logic  condition  used  a  specific  keyboard  design  (refer  to 
Section  II)  and  as  a  result,  t,.e  discussion  of  the  results  includes 
consideration  of  both  the  Logic  condition  and  the  keyboard  design  as 
potential  sources  of  the  performance  differences  observed.  This 
section  first  discusses  the  Logic/ Keyboard  Design  effects  on  keying 
accuracy,  followed  by  a  discussion  of  Logic/Keyboard  Design  effects 
on  keying  speed. 

1 .  LOGIC/KEYBOARD  EFFECTS  ON  KEYING  ACCURACY 

In  addition  to  the  results  previously  reported  for  keying  accuracy. 
Appendix  C  provides  additional  data  in  the  form  of  frequency  histo¬ 
grams  of  errors  associated  with  each  cluster  and  with  each  specific 
alphanumeric  character. 

Numeric  keying  accuracy  was  one  of  the  measures  used  to  test 
the  homogeneity  of  the  sample  population  across  Logics.  This  measure 
was  most  suited  to  this  purpose  because  of  the  constancy  of  the  keyboard 
designs  regarding  numeric  arrangement.  The  results  show  no  significant 
difference  between  the  extreme  means,  which  supports  the  homogeneity 
of  the  subjects.  The  results  also  show  the  subjects  were  following 
the  instruction  to  perform  accurately.  Further,  the  results  indicate 
the  arrangement  of  the  alphabetic  characters  on  the  keys  had  little, 
if  any,  impact  on  numeric  keying  accuracy. 
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The  results  for  numeric  keying  accuracy  are  also  consistent  with 
previously  reported  research.  Accuracy  rates  for  this  research  were 
close  to  99. 5%  correct.  Previous  studies  have  found  accuracy  rates, 
for  the  same  key  arrangement*  of  99.0%+  (Reference  8),  and  99.45% 
(Reference  7). 

Further,  the  histograms  of  Appendix  C  show  consistent  results 
regarding  the  error  frequency  of  each  numeric  character  (l.e.,  all  numeric 
characters  had  several  errors).  This  trend  Is  generally  consistent 
across  Logics  with  the  numeric  "1"  being  the  notable  exception.  For 
Logics  1,  2,  and  3,  the  number  "l"  was  located  on  the  upper-left 
corner  key  (an  easily  locatable  position)  and  experienced  no  errors 
on  any  of  the  data  collection  replications.  The  number  "1"  for  Logic 
4,  however,  was  imbedded  in  the  top  row  and  experienced  four  errors. 

This  result  illustrates  the  sensitivity  of  performance  to  design 
features  of  the  keyboard. 

The  keying  accuracy  for  the  total  list  provides  evidence  for  a 
Logic  difference  and  varying  learning  patterns.  The  most  notable  in 
both  these  qualities  is  Logic  4  which  is  not  only  significantly 
different  from  the  other  Logics  with  regard  to  proportion  correct, 
but  Logic  4  also  shows  a  fully  learned  pattern  on  all  three  Replications. 
The  other  Logics,  while  not  statistically  significant,  show  evidence 
of  improving  performance  across  the  Replications.  Examination  of  the 
histograms  of  error  frequency  by  cluster  in  Appendix  C,  shows  errors 
were  generally  made  on  the  same  clusters  for  all  Logics,  and  that  the 
source  of  difference  among  Logics  was  the  frequency  of  these  errors. 

This  indicates  a  possible  condition  that  some  clusters  were  more 
difficult  for  one  Logic  than  for  another  Logic.  Subjects  for  all  four 
Logics  reported  that  the  clusters  which  contained  both  alphabetic 
and  numeric  characters,  as  well  as  the  "less  pronounceable"  clusters 
(e.g.,  CXQCK,  HRMZ)  were  more  difficult.  These  observations  are 
generally  supported  by  the  histograms  in  Appendix  C,  which  show 
frequent  errors  for  the  coordinates  (e.g.,  N374518  W642309)  and 
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bearing/distances  (e.g.,  B166  DSP).  This  performance  was  consistent 
across  all  four  Logics,  indicating  that  these  types  of  clusters  are 
more  difficult  and  not  necessarily  made  so  by  the  Logic  2. 

Alphabetic  keying  accuracy  for  the  four  Logics  shows  basically  the 
same  pattern  as  total  list  accuracy,  which  tends  to  highlight  alphabetic 
accuracy  as  the  primary  contributor  to  list  accuracy.  This  interpretation 
is  further  strengthened  by  recalling  that  the  other  component  of  the  list, 
namely  numeric  accuracy,  was  essentially  equivalent  for  the  four  Logics 
investigated.  Logic  4,  again,  shows  a  significant  consistent  superiority 
to  the  other  Logics,  while  Logic  1  once  again  is  the  consistently  poorest 
performing  Logic.  The  histograms  of  Appendix  C  show  Logic  4  as  virtually 
error  free  {4  errors  in  3192  alphabetic  key-ins)  and  alphabetic  errors 
in  Logics  1,  2,  and  3  spread  across  essentially  all  the  characters. 

The  data  of  Appendix  C  also  show  results  consistent  with  information 
provided  cn  the  questionnaire  (Appendix  B)  regarding  the  difficulty  of 
individual  characters.  Characters  with  a  fairly  high  frequency  of 
occurrence  relative  to  the  frequency  of  other  characters  (e.g.,  G  and 
Q  in  Logic  1)  were  reported  by  several  of  the  subjects  as  the  more 
difficult  characters.  However,  some  characters  with  a  very  low  error 
frequency  (e.g.,  G,  V,  and  Z  in  Logic  2)  were  reported  as  more  difficult. 
This  contradiction  indicates  that  the  subjects  may  have  been  evaluating 
the  difficulty  of  the  character  on  some  criterion  other  than  error 
frequency.  Perhaps  they  were  associating  the  locatability  of  the 
alphabetic  character  (a  design  effect)  rather  than  the  keying  difficulty 
(a  Logic  effect).  The  apparent  insensitivity  of  the  subject  to  error 
frequency  in  identifying  difficult  characters  is  further  shown  in  the 
error  frequencies  of  "A:I  and  "B".  Whereas  these  characters  have  a  high 
relative  error  frequency  for  Logics  1,  2,  and  3,  the  subjects  using 
these  logics  typically  evaluated  "A",  "B11,  and  also  "C"  as  the  easier 
characters  to  key-in.  This  result  also  supports  the  possibility  of 
subject  sensitivity  to  keyboard  design. 
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The  data  for  the  alphabetic/numeric  transitions,  while  not  showing 
any  significant  Logic  or  Replication  effect,  did  show  accuracy  values 
consistent  with  numeric  keying  accuracy  and  alphabetic  keying  accuracy. 
This  result  indicates  the  transitions  were  not  any  more  difficult  than 
the  consecutive  numerics  or  alphabetlcs,  for  any  of  the  Logics. 

2.  LOGIC/KEYBOARD  EFFECTS  ON  KEYING  TIME 

Numeric  keying  time,  as  expected,  shows  results  consistent  with 
the  fixed  numeric  character  arrangement  across  keyboard  designs 
(Logics).  Very  slight  improvement  is  seen  in  all  Logic  conditions, 
indicating  an  early  or  previously  learned  Logic  and  design  (recall  the 
numeric  arrangement  is  that  of  the  push  button  telephone).  The  results 
reported  in  this  study  indicate  numeric  key  stroke  times  similar  to 
those  previously  reported  for  this  arrangement.  Numeric  stroke  time 
on  the  third  Replication  averaged  approximately  1.0  seconds  for  the 
four  Logics  studied,  compared  with  a  key  stroke  time  of  67  seconds 
reported  in  Reference  8.  One  possible  explanation  for  this  discrepancy 
is  the  lave!  of  learning  that  had  been  achieved  In  the  twc  studies. 

In  this  research,  the  data  show  evidence  that  learning  possibly  had  not 
been  completed,  and  therefore,  further  reduction  in  keying  time  could 
be  expected.  The  level  of  learning  for  Conrad's  data  is  unknown,  but 
expected  to  be  high. 

Total  list  keying  time  vividly  presents  the  superiority  of  Logic 
4,  as  did  total  list  keying  accuracy.  The  list  time  for  Logic  4  is 
close  to  30%  faster  than  the  other  Logics,  which  show  virtually  no 
difference  among  themselves.  Significant  learning  is  present  in  all 
Logics;  therefore,  it  Is  uncertain  as  to  the  anticipated  final  relative 
differences  among  the  Logics.  One  can  reasonably  assume,  however, 
that  Logic  4  will  continue  to  be  superior  because  of  the  fewer  number 
of  keystrokes  to  generate  an  alphabetic  character. 

One  possible  source  of  the  Logic  effect  was  identified  as  the 
accuracy  rate  associated  with  each  Logic.  The  rationale  is  that  more 
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errors  will  require  more  corrections  and  thus  take  longer  to  key- In 
the  list.  To  test  this  possibility.  Calculated  List  Time  (CLTJME)  was 
obtained  from  the  MTIME  and  AST1ME  data  for  each  Logic  (these  values 
were  derived  from  the  error-free  key-ins  and  thus  represent  error-free 
performance).  The  mean  CLTIME  for  eacn  Logic  and  Replication  is 
plotted  in  Figure  28.  This  plot,  when  compared  with  that  in  Figure 
23,  shows  virtually  identical  trends  and  relationships  between  Logics. 

Similarly,  as  before,  a  two-way  (Logic  by  Replication)  ANOVA  shows 
a  significant  Logic  effect,  ^(3,6)  =  39.95,  p  <  .0002,  and  a  significant 
Replication  effect,  ^(2,6)  =•  12.17,  p  <  .007.  Duncan's  Multiple  Range 
Test  on  the  Logics  reveals,  as  before,  a  significant  difference  (a  = 

,05,  df  =  6)  between  Logic  4  and  the  other  three  Logics,  and  no 
differences  among  Logics  1,  2,  and  3.  This  result  strongly  contradicts 
the  possibility  that  keying  accuracy  is  the  scarce  of  this  effect. 

Another  possible  source  will  be  aiscussed  shortly,  in  conjunction  with 
alphabetic  keying  time. 

From  the  results  presented  previously,  alphabetic  keying  time 
appears  to  be  the  primary  measure  accounting  for  the  observed  differences 
in  total  list  time.  As  the  results  indicate,  the  time  to  key  an 
alphabetic  character  with  Logic  4  is  approximately  twice  as  fast 
as  the  time  with  Logic  1,  2,  or  3.  This  basic  time  difference  is 
easily  accounted  for  by  the  differences  in  Logic  structure.  Logics 
1  and  2  require  three  key  strokes  to  generate  an  alphabetic  character. 
Logic  3  requires  two  or  three  key  strokes.  Logic  4  requires  only  one 
key  stroke. 

In  order  to  equalize  this  difference  between  Logics,  alphabetic 
stroke  time  was  extracted  from  the  data  and  noticeably  different 
results  are  obtained.  Logic  1  now  is  observed  to  have  the  fastest 
stroke  time  and  Logic  4  now  has  the  slowest.  Interestingly ,  this 
implies  the  subject  is  actually  keying  faster  with  Logics  1  and  2, 
but  performing  worse  in  the  aggregation  because  more  (3  times) 
strokes  are  required  to  generate  an  equivalent  character.  Thus,  it 
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appears  this  basic  Logic  difference  Is  the  source  of  the  Logic  effect 
observed  in  both  total  lHt  time  and  alphabetic  keying  time 

Differences  among  Logics  can  alsc  be  accounted  for  using  the 
stimulus-response  paradigm  (Reference  20).  If  the  keying  task  used 
in  this  research  Is  structured  into  a  discrimination  (stimulus  Input) 
stage,  mediation  stage,  and  response  execution  stage.  Logics  1  and  2 
loop  through  the  latter  two  stages  three  times  before  an  alphabetic 
character  is  generated;  Logic  3  loops  through  two  or  three  times;  and 
Logic  1  only  passes  through  these  stages  once.  Thus,  since  there  is 
a  time  factor  involved  in  each  of  these  stages.  Logic  4  would  obviously 
be  the  fastest. 

As  one  further  evaluation  of  the  keying  tiY.e  data,  methods-time 
measurement  (MTM)  tables  were  consulted  (Reference  21)  and  estimated 
times  derived  for  a  cluster  from  the  data  collection  list:  '2345'. 

The  MTM  values  were  calculated  by  using  tabled  values  for  a  reach 
involving  finger  motion  to  a  small  object  requiring  accuracy,  a 
contact  grasp,  and  a  pressure  activation.  Based  on  this  approach, 

MTM  predicted  a  keying  time  of  2.46  seconds  for  '2345'  using  Logic  1. 
This  MTM  value  is  consistent  with  Conrad’s  data  (predicting  a  time 
of  2.68  seconds),  and  the  data  from  this  experiment  for  Logic  1  of 
2.50  seconds  for  the  string  ' 23^  . 
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SECTION  V 

CONCLUSIONS  AND  RECOMMENDATIONS 

Conclusions  supported  by  the  results  of  this  research  will  be 
stated.  Subsequently,  recommendations  regarding  an  Improved  keyboard 
design  based  on  subjects'  comments  (Appendix  B)  will  be  provided,  along 
with  recommendations  regarding  further  research. 

1.  CONCLUSIONS 

Of  the  Logics  investigated,  one-hand  keying  of  alphanumeric 
clusters  can  be  accomplished  most  accurately  and  fastest  with  Logic 
4  and  a  suitable  keyboard  designed  for  one-hand  operation.  The  data 
summarized  in  Table  4  indicate  that  this  Logic  is  fastest  and  most 
accurate  when  compared  to  other  viable  Logics  for  keying  alphanumeric 
characters.  The  tabularized  data  also  show  the  primary  difference 
between  Logic  4  and  the  other  three  Logics  to  be  time  oriented.  The 
time-related  dependent  variables  were  all  significant,  with  the 
exception  of  numeric  keying  time  which  was  expected  to  be  non¬ 
significant.  Further,  previously  presented  data  show  Logic  4  to  be 
essentially  learned  from  the  start  of  the  data  collection.  The  other 
Logics,  most  notably  Logic  1,  not  only  exhibit  a  more  pronounced 
learning  trend,  but  also  had  yet  to  reach  the  level  of  initial 
performance  for  Logic  4  (even  after  three  Replications). 

Regarding  the  three  Logics  which  used  the  same  keys  for  both 
the  alphabetic  and  numeric  characters.  Logic  3  which  utilized  two 
"ALPHA"  keys  appears  the  best  performer.  However,  the  apparent  reason 
for  Logic  3's  superiority  is  that  fewer  key  strokes  are  required  to 
generate  an  alphabetic  character. 

It  also  has  to  be  concluded  that  some  additional  performance 
improvement  is  possible  for  all  Logics,  and  therefore,  performance 
values  for  the  learned  user  remains  questionable,  but  is  expected  to 
approach  those  achieved  in  other  referenced  studies. 
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2.  DESIGN  RECOMMENDATIONS 

Logic  4  and  a  keyboard  design  comparable  to  that  used  in  this 
research  are  recommended,  based  on  the  results  observed  under  the 
experimental  conditions  of  this  research.  With  regard  to  the 
Implementation  of  this  Logic,  the  subjects  offered  no  better 
alternative  and  rated  Logic  4  "Not  Difficult".  The  keyboard  design 
used  with  Logic  4  received  minor  criticism  regarding  the  location 
of  the  "Z".  To  remain  consistent  with  the  pattern,  the  "Z"  should 
probably  be  placed  below  the  "U". 

If  space  is  such  a  critical  factor  that  a  keyboard  of  this  type 
is  Impractical,  based  on  the  results  of  this  research  a  design  which 
utilizes  Logic  3  but  has  the  l',yout  of  the  keyboard  used  with  Logic  1 
and  2  is  recommended.  Additionally,  a  third  "ALPHA"  key  should  be 
added  to  designate  the  center  alphabetic  character.  The  location 
of  the  "ALPHA"  keys  should  also  be  changed,  according  to  subject 
opinions.  Many  subjects  felt  the  "ALPHA"  key  was  visually  and 
tactilely  imbedded  among  the  other  keys.  Subjects  recommended  either 
a  location  or  shape  change  to  give  prominence  to  these  keys.  \ 
recommended  keyboard  design  incorporating  these  improvements  is 
conceptualized  in  Figure  29. 

3.  RESEARCH  RECOMMENDATIONS 

The  results  show  learning  is  apparently  still  occurring  after 
the  three  Replications  used  in  this  research.  Based  on  this  obser¬ 
vation,  further  research  is  recommended  which  concentrates  or.  subject 
performances  after  the  data  show  a  fully  learned  condition.  Initially, 
such  a  study  would  be  informative  for  both  the  superior  Logic  4/Keyboard  C 
design  of  this  research,  and  the  recommended  Logic/Keyboard  design  of 
Figure  29.  It  would  be  expected  that,  with  enough  learning,  key 
stroke  time  could  be  reduced  to  closely  approximate  that  found  by 
Conrad  and  others,  with  accuracy  remaining  close  to  the  99.5%  correct 
value  found  in  this  research,  and  by  Conrad  and  others. 
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Figure  29.  Recommended  4  *  3  Matrix  Keyboard  Design 


67 


'  / 


'I 

7 

j 

i 


j 


■j 


ttoitea  v  a  fM-ttff  iftr  i>irr  :>  i  a 


B 


Another  research  recommendation  already  alluded  to  Is  the  study 
of  keying  accuracy  and  keying  speed  performance  for  the  Log 1c/ Keyboard 
design  recommended  earlier  In  this  Section.  Based  on  observed 
performance  with  Logic  3  and  subjective  comments  regarding  the  design 
of  Keyboard  A  (Logic  1  and  2  Keyboard),  It  Is  expected  that  keying 
performance  would  be  superior  to  that  of  Logic  3  and  perhaps  even 
approach  the  keying  performance  of  Logic  4. 

Also,  further  research  Is  recommended  which  examines  performance  with 
the  alternative  Logics  under  more  realistic  experimental  conditions. 

First,  the  keying  task  should  be  sup  lemented  with  other  realistic 
tasks  to  create  the  divided  attention  situation  under  which  operation 
of  the  keyboard  is  likely  to  occur.  A  tracking  task,  visual  search  task, 
and/or  an  auditory  recognition/verbal  response  task  would  be  candidate 
auxiliary  tasks  representative  of  typical  flight  crew  tasks.  Also, 
since  military  pilots  fly  with  gloves  on,  the  effect,  if  any,  gloves 
have  on  keying  performance  needs  to  be  ascertained.  Along  with  these, 
the  subjects  should,  more  properly,  be  pilots,  who  are  trained  to 
the  divided  attention  nature  of  commanding  an  aircraft. 
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APPENDIX  A 

INSTRUCTIONS  TO  SUBJECTS 


1.  GENERAL  INFORMATION 

The  study  in  which  you  are  about  to  participate,  examines  performance 
differences  associated  with  various  alphanumeric  keyboard  designs.  In  the 
course  of  your  participation  today,  you  will  be  trained  on  the  use  of  an 
alphanumeric  keyboard,  you  will  be  allowed  to  practice  with  the  keyboard, 
and  then  I  will  ask  you  to  create  a  list  on  the  CRT  using  the  keyboard. 

At  this  time,  I  will  explain  the  keyboard  you  are  about  to  use  and  the 
method  you  must  use  to  get  a  numeric  or  alphabetic  character.  .  .  . 

a.  Keyboard  A/ Logic  1 

As  you  notice,  this  arrangement  is  a  familiar  one.  Tne  numbers  are 
arranged  in  the  same  location  as  they  are  on  pushbutton  telephones  (show 
location  ot  1,  2,  3,  4,  5,  6,  7,  8,  9,  0).  The  alphabet  has  been 
assigned  to  the  keys  1  through  S  with  three  letters  on  each  key.  The 
exception  Is  the  last  key,  which  has  only  the  two  letters  Y  and  Z. 

Notice  also,  there  is  a  left,  center,  and  right  position  for  a  letter 
on  the  kev,  thus  A  is  left,  8  is  center,  and  C  is  right;  Y  is  left, 

Z  is  center,  and  there  is  nothing  in  the  right  position  on  this  (the  9) 
key. 


Further,  for  this  keyboard,  notice  there  are  two  inactive  keys  and 
six  special  function  keys  (show  the  six).  I'll  explain  these  special 
function  keys  as  we  go  along. 

To  select  a  number,  you  simply  depress  the  keys  1  through  0, 
corresponding  to  the  desired  number  1,  2,  3,  4,  5,  6,  7,  8,  9,  0 
(demonstrate  on  CRT). 

To  select  a  letter,  you  must  depress  three  keys  in  sequence. 

The  first  key  you  depress  will  always  be  the  "ALPHA  FUNCTION"  key. 
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The  second  and  third  key  you  depress  depends  upon  which  letter  you 
want.  If  you  want  a  letter  on  the  "5"  key,  you  depress  the  "5"  key 
second;  then  If  It  Is  specifically  the  M  you  want,  depress  the  "4" 
key  third;  if  it  is  the  N  you  want,  you  depress  the  "5"  key  third; 

If  It  is  the  0  you  want,  depress  the  "6"  key  third.  The  general 
method  for  selecting  a  letter  Is  to: 

1.  Depress  the  "ALPHA  FUNCTION". 

2.  Depress  the  key  which  has  the  letter  on  it  which  you  want. 

3.  Depress  the  left,  center,  or  right  key  In  that  row  of 
three  keys  which  corresponds  to  the  left,  center,  or  right 
position  of  the  desired  letter. 

I  will  now  demonstrate:  1,  2,  3,  BRAVO  456.  Now  you  create  this 
same  string  (Subject  keys  alphabetic/numeric  string  while  experimenter 
observes) . 

Are  there  any  questions? 

b.  Keyboard  A/ Logic  2 

As  you  notice,  this  arrangement  is  a  familiar  one.  The  numbers 
are  arranged  in  the  same  location  as  they  are  on  pushbutton  telephones 
(show  location  of  1,  2,  3,  4,  5,  6,  7,  8,  9,  0).  The  alphabet  has 
been  assigned  to  the  keys  1  through  9  with  three  letters  on  each  key, 
the  exception  is  the  last  key,  which  has  only  the  two  letters  Y  and  Z. 
Notice,  there  is  a  first  (or  left),  second  (or  center),  and  third 
(or  right)  position  for  a  letter  on  the  key,  thus  A  is  first,  B  is 
second,  and  C  is  tnird;  Y  is  first,  l  is  second,  and  there  is  nothing 
in  the  third  position  on  this  key. 

Further,  for  this  keyboard,  notice  there  are  two  inactive  keys 
and  six  special  function  keys  (show  the  six).  I'll  explain  these 
special  function  key:  as  we  go  along. 
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To  select  a  number,  you  simply  depress  the  key  1  through  0, 
corresponding  to  the  desired  number  1,  2,  3,  4,  5,  6,  7,  8,  9,  0 
(demonstrate  on  keyboard). 

To  select  a  letter,  you  must  depress  three  keys  in  sequence. 

The  first  key  you  depress  will  always  be  the  "ALPHA  FUNCTION"  key.  The 
second  and  third  key  you  depress  depends  upon  which  letter  you  want. 

If  you  want  a  letter  on  the  "5"  key,  you  depress  the  "5"  key  second; 
then  if  It  is  specifically  the  M  you  want,  depress  the  ”1"  key  third; 

If  it  is  the  N  you  want,  you  depress  the  "2"  key  third;  if  it  is  the 
0  you  want,  depress  the  "3"  key  third.  The  general  method  for  selecting 
a  letter  Is  to: 

1 .  Depress  the  "ALPHA  FUNCTION" . 

2.  Depress  the  key  which  has  the  letter  on  it  which  you  want. 

3.  Depress  the  "i",  "2",  or  "3"  key  which  corresponds  to 

the  left,  center,  or  right  position  of  the  desired  letter. 

I  will  now  demonstrate:  1,  2,  3,  BRAVO  456.  Now  you  create  this 
same  string  (Subject  keys  alphabetic/numeric  string  while  experimenter 
observes). 

Are  there  any  questions? 

c.  Keyboard  8/ Logic  3 

As  you  can  see,  this  arrangement  is  somewhat  unfamiliar.  While 
the  numbers  are  arranged  in  the  same  location  as  they  appear  on  push¬ 
button  telephones  (show  location  of  1 ,  2,  3,  4,  5,  6,  7,  8,  9,  0),  the 
letters  have  been  assigned  keys  2  through  0,  with  three  letters  on 
each  key,  the  exception  is  the  "4"  key  which  has  only  the  two  letters, 
"A"  and  "R".  Notice,  there  is  a  left,  center,  and  right  position  for 
a  letter  on  a  key.  Thus,  "A"  is  in  a  left  position,  "B"  is  in  a  center 
position,  anu  there  is  nothing  in  the  right  position  on  the  "4"  key. 
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Further,  notice  that  for  this  keyboard,  there  Is  one  Inactive  key  and 
seven  special  function  keys  (show  the  seven).  I'll  explain  these 
special  function  keys  as  we  go  along. 

To  select  a  number  you  simply  depress  the  key  "1"  through  "0", 
corresponding  to  the  desired  number  1,  2,  3,  4,  5,  6,  7,  8,  9,  0  (demon¬ 
strated  on  keyboard). 

To  select  a  letter,  you  must  depress  either  two  or  three  keys  In 
sequence,  depending  upon  the  letter  desired.  A  letter  in  the  left 
position  requires  two  keys  -  the  ALPHA  LEFT  first,  followed  by  the  key 
upon  which  the  desired  letter  is  located  (I.e.,  To  select  an  "A", 
you  first  depress  ALPHA  LEFT  and  then  depress  the  "4"  key).  Letters 
in  the  right  position  require  a  similar  action,  only  the  ALPHA  RIGHT 
key  is  depressed  Instead  of  the  ALPHA  LEFT,  (i.e..  To  select  a  "K", 
you  first  depress  ALPHA  RIGHT  and  then  depress  the  "5"  key).  A 
letter  in  the  center  position  requires  three  keys  -  the  ALPHA  LEFT, 
then  the  ALPHA  RIGHT,  then  the  key  upon  which  the  desired  character 
is  located  (i.e..  To  select  an  "M",  you  first  depress  ALPHA  LEFT,  then 
ALPHA  RIGHT,  and  then  the  "8"  key).  The  general  method,  again,  is  to: 

1.  Depress  either  the  ALPHA  LEFT  alone,  the  ALPHA  RIGHT  alone, 
or  the  ALPHA  LEFT  followed  by  the  ALPHA  RIGHT. 

2.  Depress  the  key  which  has  the  letter  on  it  which  is  desired. 

I  will  now  demonstrate:  1,  2,  3,  BRAVO  456.  Nov*  you  create  this 
same  string  (Subject  keys  alphabetic/numeric  string  while  experimenter 
observes) . 

Are  there  any  questions  to  this  point? 

d.  Keyboard  C/Logic  4 

As  you  can  see,  this  arrangement,  while  not  familiar,  is  simply  a 
matrix  of  letters  and  numbers  in  sequence.  Notice  further,  that  there 
are  seven  inactive  keys  and  five  special  function  keys  (point  out  the 
five).  I'll  explain  their  purpose  as  we  go  along. 
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To  select  a  number,  you  simply  depress  the  key  1  through  0, 
corresponding  to  the  desired  number  1,  2,  3,  4,  5,  6,  7,  8,  9,  0 
(demonstrate). 

To  select  a  letter,  you  also  simply  depress  the  key  A  through  Z, 
corres porting  to  the  desired  letter  A,  B,  C,  D,  E,  F,  G,  H,  I,  J,  K, 

L,  M,  N,  0,  P,  Q,  R,  S,  T,  U,  V,  W,  X,  Y,  Z  (demonstrate) .  I  will 
now  demonstrate:  1,  2,  3,  BRAVO  456.  Now  you  create  this  same  string 
(Subject  keys  alphabetic/numeric  string  while  experimenter  observes). 

Are  there  any  questions  to  this  point? 

2.  LIST  AND  DISPLAY  INFORMATION 

Before  ycu  practice,  let  me  explain  the  list  format,  display 
format,  and  the  special  function  keys. 

The  list  is  formatted  in  three  columns,  l.v  rows.  In  other  words, 
you  read  across  the  columns  a  row  at  a  time.  As  you  key- in  the 
letters  and  numbers,  they  will  appear  below  the  dashed  line,  starting 
where  the  cursor  is  now  located  and  proceeding  left  to  right.  This 
"Scratch  Pad"  portion  of  the  CRT  is  made  up  of  three  lines,  one  for 
each  of  the  columned  clusters  in  the  row  you  are  keying-in.  You 
key  the  row  into  the  Scratch  Pad  in  the  following  manner:  ABC/12335, 
point  out  errors,  back  space,  correct  to  12345,  forward  space, /XYZ. 

When  you've  completed  a  row  from  the  list,  depress  ENTER,  which  clears 
the  scratch  pad,  and  returns  the  cursor  to  the  starting  point  for  the 
next  row.  Notice  that  the  gets  used  after  the  first  two  lines 
in  the  Scratch  Pad  and  the  ENTER  gets  used  only  after  the  third  line 
when  you  are  ready  to  clear  the  Scratch  Pad  and  start  the  next  row  on 
the  list.  Also,  to  re-emphasize,  errors  can  only  be  corrected  while 
in  the  Scratch  Pad,  and  only  while  you  are  in  the  same  line.  Once 
you've  gone  on  to  the  next  Scratch  Pad  line,  or  entered  the  Scratch 
Pad  lines,  you  can  not  go  back  and  correct. 
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I've  demonstrated  the  keyboard  and  the  method  for  keying-in  letters 
and  numbers  in  the  Scratch  Pad,  I've  demonstrated  the  special  functions 
for  back-spacing,  forward-spacing,  clearing  an  entry,  and  entering. 

Are  there  any  questions  now,  before  you  practice? 

3.  PRACTICE  INSTRUCTIONS 

This  is  the  list  for  you  to  practice  with.  After  you  key-in  all 
the  rows  of  the  list,  continue  by  simply  starting  over  at  the  top  of 
the  list.  Your  practice  will  continue  until  you  have  keyed-in  the 
list  twice  in  a  row,  without  making  an  error  (After  subject  completes 
practice  session,  allow  a  5-10  minute  break  before  proceeding  to  the 
data  collection  session). 

4.  rATA  COLLECTION  INSTRUCTIONS 

Here  is  the  list  I  would  now  like  you  to  key-in  using  this  keyboard. 
Remember,  accuracy  counts  as  well  as  speed.  Therefore,  key-in  this  list 
at  a  comfortable  pace,  making  sure  that  the  lines  in  the  Scratch  Pad 
are  error-free  before  they  are  entered.  I  vill  ask  you  to  key-in  this 
list  three  times,  however,  wait  for  my  instruction  each  time  through 
the  list.  Do  you  have  any  questions  at  this  time?  You  may  proceed 
when  you  are  ready. 


74 


1  / 


—  -***c 


AFWAL-TR-81  -31 04 


APPENDIX  B 

KEYBOARD/ LOGIC  QUESTIONNAIRE 
and 

SUMMARY  OF  RESULTS 


TABLE  OF  CONTENTS 

_ HEM _ 

Subject  Questionnaire 
Summary  of  Subjects'  Comments  (Logic  1) 
Summary  of  Subjects'  Comments  (Logic  2) 
Summary  of  Subjects'  Comments  (Logic  3) 
Summary  of  Subjects'  Comments  (Logic  4) 
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KEYBOARD/LOGIC  QUESTIONNAIRE 

1*  you  make  any  errors?  How  Many? 

Rep  1  _  _ 

Rep  2  _  _ 

Rep  3 _  _ 

2.  Did  you  correct  all  errors?  Estimated  Accuracy 

Rep  1  _  _ _ 

Rep  2  _  _ _ 

Rep  3  _  _ 

3.  This  method  of  keying-in  letters  and  numbers  is 

Not  Moderately  Very 

Difficult  Difficult  Difficult 

□  □  □  □  □ 

4a.  Where  any  characters  more  difficult  to  key-in  than  the  others? 
Which  ones? 


4b.  Where  any  characters  easier  to  key-in  than  others? 
Which  ones? 
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5a.  Were  any  clusters  more  difficult  to  key-in  than  any  othe 
Which  ones? 


5b.  Were  any  clusters  easier  to  key-in  than  others? 
Which  ones? 


6.  Do  you  have  a  pushbutton  telephone? 
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Summary  of  Subjects'  Comments  for  Logic  1 


Method 

Not 

was : 

Difficult 

Moderately  Difficult 

Very  Difficult 

m 

m  pn  n. 

U  □ 

Easier 

Characters : 

-  Letters  requiring  same  key; 

e.g. ,  A,N,R,  etc. 

Harder 

Characters: 

-  Alphabet  extremes 

-  Q,U,Z,G,H 

Easier 

Clusters : 

-  None 

-  All  letters 

-  FL,  ABC,  TWINE,  HUDQLS,  MUPPY, 

JUVTY 

Harder 

Clusters ; 

-  Number  strings 

-  CXQCK,  HRMZ 

-  Coordinates 

-  None 

Comments:  —  Alpha  key  imbedded,  prefer  relocation 

-  Prefer  to  have  "Alpha  Hold"  to  key  consecutive 

letters  without  having  to  key  ALPHA  each  time 

—  Relocate  special  functions  above  4x3  matrix 

-  Prefer  less  key-hits  per  letter 
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Summary  of  Subjects'  Conments  for  Logic  2 


Method  was: 

Not  Difficult  Moderately  Difficult  Very  Difficult 

□  mm  mm 

Easier  Characters:  -  None 

-  Letters  requiring  same  key-hit;  e.g.  A,E,I 

-  1,2,3 

Harder  Characters:  -  Z,D,G,Q,V 

-  None 

Easier  Clusters:  FL,  TWINE,  FOXX,  ABC 
Harder  Clusters:  -  CXQCK,  HRMZ 

Number  strings 

-  Ones  unpronouncable 

-  Coordinates 

Comments:  -  Relocate  ALPHA  key,  or  shape  code 

-  Three  key-hits  per  letter  too  many,  2  might  be 

acceptable 

-  Logic  was  not  difficult  to  learn 

—  Key  for  each  letter  would  be  faster 
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Summary  of  Subjects'  Comments  for  Logic  3 

Method  was: 

Not  Difficult  Moderately  Difficult  Very  Difficult 

CD  m  CD  □  CD 

Easier  Characters:  -  F,L,A,Z 

-  Letters  in  easy  key  positions 

-  Vowels 

-  Numerics 

Harder  Characters:  -  Q>X,B,N,S,E,W,Z 

-  Letters  not  used  very  much 
Letters  in  middle  of  subsets 

Easier  Clusters:  -  ABC,  TWINE 

-  Coordinates 

-  12345 

-  Repeated  Clusters  (e.g.,  FL) 

Harder  Clusters:  -  1IUQLS ,  CXQCK,  HRMZ,  VGJ,  JUVTT 

-  Non-phonetics 

Comments:  -  Relocate  ALPHA  keys 

—  Shape  code  ALPHA  keys 

-  Add  key  for  ALPHA  Center 

-  Separate  backspace  and  forward  space  from  ALPHAS 
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Summary  of  Subjects'  Comments  for  Logic  4 


Method  was: 

Not  Difficult  Moderately  Difficult  Very  Difficult 

GD  DD  □  QD  □ 

Easier  Characters:  -  A,Z,B,C 

-  Numbers 


Harder  Characters:  -  Z, J,V,W,X,F,G 

Easier  Clusters:  -  ABC,  123,  ABZ,  ABQ,  FL,  MUPPY 

-  Phonetics 

-  Number  strings 

Harder  Clusters:  -  Latitudes/Longitudes 

-  Ones  unpronouncable 


-  HUQLS,  CXQCK,  VGJ,  RGS 

Comments:  -  More  space  separa'  xon  between  alphabetics  and  numerics 

-  "Z"  out  of  place;  put  under  the  ”U" 

-  Forward  space  imbedded 

-  Method  difficult  because  familiar  with  "QWERTY" 

arrangement . 

-  Key  pressure  caused  fatigue  after  long  list 
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